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FOREWORD 


This  research  and  development  was  performed  in  response  to  recognized  Navy  needs 
for  an  investigation  of  the  effects  of  initial  training  and  job  conditions  on  skill  retention 
under  subproject  RF63-522-801-014  (Computer-aided  and  Classroom  Training),  work  unit 
03.03  (Still  and  Knowledge  Retention).  In  previous  work  conducted  to  investigate 
relationships,  problems  were  observed  in  the  retention  of  knowledge  and  skills  required  to 
perform  acoustic  analysis. 

The  objectives  of  the  subproject  are  to  (1)  derive  ways  of  detecting  potential 
problems  existing  in  the  Navy  and  (2)  recommend  means  to  minimize  performance 
deterioration  by  restructuring  training  and  job  conditions.  The  objective  of  this  effort 
was  to  evaluate  the  methods  used  to  train  and  test  aviation  antisubmarine  warfare 
operators  (AWs)  in  the  complex  skill  of  passive  acoustic  analysis  and  classification. 
Results  are  intended  primarily  for  the  Anti-Submarine  Warfare  Wing,  Pacific,  the  Fleet 
Aviation  Specialized  Operational  Training  Group,  Pacific  (FASOTRAGRUPAC),  and  other 
agencies  at  FASOTRAGRUPAC  concerned  with  passive  acoustic  sonar  operator  training 
and  testing.  They  should  also  be  of  interest  to  agencies  responsible  for  the  development 
of  skill  acquisition  and  maintenance  programs  for  personnel  in  other  technical  ratings. 

Appreciation  is  expressed  to  the  members  of  the  instructor  staff  at  the  AW  Common 
Core  Acoustic  Analysis  School,  FASOTRAGRUPAC,  for  their  time  and  efforts  in 
collecting  material  for  this  report.  Particular  appreciation  is  expressed  to  AWCS  Terry 
Burkette,  AWC  Henry  Stanley,  and  Mr.  Paul  Lawson  for  their  support. 


3.  W.  RENARD 
Commanding  Officer 


JAMES  W.  TWEEDDALE 
Technical  Director 
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SUMMARY 


Problem 


Technical  personnel  in  the  Navy  are  required  to  perform  tasks  that  range  in  difficulty 
from  simple,  ordered  procedural  tasks  to  highly  complex  cognitive  tasks.  As  the  Navy 
becomes  more  reliant  upon  sophisticated,  complex  defense  systems  to  meet  mission 
requirements,  there  is  an  increased  need  to  improve  instruction  designed  to  teach 
operators  and  technicians  to  perform  the  complex  cognitive  tasks  required  in  their  jobs. 
However,  training  in  most  Navy  technical  schools  still  relies  on  instructional  methods 
designed  for  teaching  procedural  tasks.  Aviation  antisubmarine  warfare  operators  (AWs) 
destined  to  be  assigned  to  Pacific  Fleet  carriers  receive  training  in  the  complex  task  of 
acoustic  analysis  and  classification  at  the  AW  Common  Core  Acoustic  Analysis  School, 
Fleet  Aviation  Specialized  Operational  Group,  Pacific  (FASOTRAGRUPAC).  The  quality 
of  instructional  and  testing  methods  used  to  teach  this  complex  task  needs  to  be 
evaluated. 

Objective 

The  objective  of  this  effort  was  to  evaluate  the  quality  of  instructional  methods  used 
to  teach  and  test  the  complex  task  of  acoustic  analysis  and  classification  at  the  Common 
Core  course,  and  to  provide  recommendations  for  course  changes  that  would  improve 
initial  acquisition  and  long-term  retention  of  the  skills  and  knowledge  required  to  perform 
that  task. 

Approach 

The  teaching  and  testing  methods  used  at  the  Common  Core  course  were  evaluated  in 
terms  of  three  criteria:  (1)  structural  organization  of  the  course,  (2)  quality  of  feedback 
and  remediation,  and  (3)  testing  methods  used  to  evaluate  student  performance. 

Results 


1.  The  structural  organization  of  the  course  was  found  to  have  four  deficiencies: 
(a)  no  clear  conceptual  model  of  the  task,  (b)  an  overemphasis  on  learning  discrete  facts 
that  requires  large  amounts  of  rote  memorization,  (c)  two  topic  areas  presented  in  the 
course  that  are  not  practiced  or  tested  as  skills,  and  (d)  terminal  learning  objectives  that 
do  not  require  the  students  to  perform  the  task. 

2.  The  quality  of  feedback  and  remediation  was  found  to  have  three  deficiencies: 
(a)  the  quality  of  diagnostic  feedback  for  repeated  errors  in  subprocedures  was  inade¬ 
quate,  (b)  students  have  no  way  of  evaluating  their  own  performance,  in  large  part 
because  of  the  lack  of  an  appropriate  conceptual  model  of  the  task,  and  (c)  there  is  no 
systematic  approach  to  remediating  poorly  understood  knowledge  or  procedural  skills. 

3.  Testing  methods  were  found  to  be  deficient  in  testing  knowledge  factors 
primarily  because  of  the  emphasis  on  recognition  of  discrete  facts  rather  than  on  the 
conceptual  understanding  of  fundamental  relationships  required  by  the  task. 

Conclusions 


1.  Although  this  evaluation  revealed  a  number  of  deficiencies  in  course  organiza¬ 
tion,  diagnostic  feedback,  and  testing  methods,  instructor  attitude  was  not  cited  as  a 


problem.  It  is  very  likely  that  these  deficiencies  have  been  in  part  overcome  by  individual 
instructor  efforts.  However,  relying  on  this  type  of  unsystematic  approach  cannot  ensure 
high  quality  graduates  of  this  school. 

2.  The  course,  as  it  is  presently  being  taught,  does  not  explicitly  provide  the 
student  with  a  qualitative  understanding  of  the  relationship  between  sound  source  genera¬ 
tors  and  their  representations  on  the  lofargram.  The  development  and  use  of  a  model 
should  provide  a  better  conceptual  understanding  of  this  relationship,  aid  the  student  in 
the  initial  acquisition  of  the  skills  and  knowledge  underlying  acoustic  analysis  and 
classification  techniques,  provide  a  better  structure  for  the  retention  of  the  procedural 
skills  and  the  memorized  acoustic  intelligence  data  base,  and  aid  instructors  in  curriculum 
development,  classroom  demonstrations,  and  test  development. 

3.  Although  the  student  is  provided  with  feedback  about  performance  on  knowledge 
and  procedural  skills,  little  individual  feedback  is  directed  toward  diagnosing  and  correct¬ 
ing  repeated  errors  caused  by  conceptual  misunderstandings.  In  addition,  remediation 
material  that  specifically  addresses  the  errors  made  by  students  in  classroom  work  or  on 
tests  is  needed. 

4.  Tests  of  knowledge  factors  do  not  presently  require  the  students  to  demonstrate 
conceptual  understanding  of  the  material  but,  rather,  to  employ  rote  memorization  tech¬ 
niques  that  may  mask  inadequate  conceptual  understanding  of  the  task. 

Recommendations 

It  is  recommended  that  FASOTRAGRUPAC  support  the  following  actions: 

1.  Improve  the  course  by  development  and  use  of  a  conceptual  model  designed  to 
increase  the  student's  understanding  of  the  relationship  between  sound  source  generators 
and  their  representation  of  the  lofargram.  This  model  can  be  fully  integrated  into  the 
course  curriculum. 

2.  Ensure  that  information  presented  in  the  oceanography  and  sonobuoy  segments  of 
the  course  is  taught,  practiced,  and  tested  as  skills  needed  to  perform  acoustic  analysis 
and  classification.  If  practical  application  of  that  information  cannot  be  accomplished  in 
this  course,  those  segments  may  be  moved  to  a  more  appropriate  point  in  the  training 
pipeline. 

3.  Ensure  that  course  objectives  require  students  to  perform  the  skill  of  acoustic 
analysis  and  classification  on  the  lofargram. 

4.  Ensure  that  instructors  provide  diagnostic  feedback  to  individual  students  about 
conceptual  errors  in  their  understanding  of  the  task,  as  evidenced  by  repeated  errors  in 
subprocedures. 

5.  Ensure  that  remediation  of  students  who  perform  poorly  on  classroom  work  or  on 
tests  is  directed  toward  correcting  the  specific  poor  performance  rather  than  requiring 
the  students  to  study  new  material  for  a  longer  period  of  time. 

6.  Ensure  that  tests  assess  the  student's  level  of  conceptual  understanding  of  the 
materials  presented  in  the  course,  and  that  test  questions  assess  the  student's  ability  to 
recall  rather  than  recognize  factual  information. 
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INTRODUCTION 


Problem  and  Background 


Navy  technical  personnel  are  required  to  perform  tasks  that  range  in  difficulty  from 
simple,  ordered  procedural  tasks  to  highly  complex,  cognitive  tasks.  Training  in  most 
Navy  technical  schools  still  relies  on  instructional  methods  designed  for  teaching 
uncomplicated  procedural  tasks.  As  the  Navy  becomes  more  reliant  upon  sophisticated, 
complex  defense  systems  to  meet  mission  requirements,  there  is  an  increased  need  to 
improve  instruction  designed  to  teach  operators  and  technicians  to  perform  the  complex, 
cognitive  tasks  required  in  their  jobs. 


The  training  methods  presently  used  in  many  Navy  technical  courses  are  not  adequate 
to  prepare  technical  personnel  to  operate  and  maintain  these  complex  systems  under  the 
wide  variety  of  conditions  encountered  in  the  operational  environment.  Technical 
personnel  are  expected  to  learn  difficult  subject  areas,  perform  multiple  complex 
procedures,  and  memorize  large  factual  data  bases  through  courses  that  are  poorly 
organized,  overly  detailed,  and  not  designed  to  teach  or  test  cognitively  complex  tasks. 
This  approach  to  teaching  ignores  important  relationships  between  topics,  does  not 
provide  a  good  conceptual  model  of  the  task,  and  emphasizes  recognition  of  facts  rather 
than  understanding  of  principles.  Because  of  these  failings,  it  is  likely  that  trainees  will 
not  be  able  to  understand  the  cognitively  complex  material  presented  completely  and  that 
their  retention  of  the  skills  and  knowledge  learned  will  be  poor. 


Aviation  antisubmarine  warfare  operators  (AWs)  assigned  to  the  aviation  community's 
S-3A  platform  and  destined  for  assignment  to  the  Pacific  Fleet  report  to  the  Fleet 
Aviation  Specialized  Operational  Training  Group,  Pacific  (FASOTRAGRUPAC),  Naval  Air 
Station,  North  Island.  San  Diego,  California  for  20  weeks  of  training  after  they  graduate 
from  AW  "A"  school.  The  first  4  weeks  are  spent  in  the  AW  Common  Core  Acoustic 
Analyses  Course,  where  AWs  receive  training  in  the  complex  cognitive  tasks  of  acoustic 
analysis,  integration  of  acoustic  intelligence  information,  and  target  classification  pro¬ 
cedures.  In  previous  research  on  the  retention  of  procedures  and  knowledge  gained  in  that 
course,  Wetzel,  Konoske,  and  Montague  (1983)  found  that  AWs'  analysis  skills  and  memory 
of  acoustic  intelligence  degraded  significantly  within  4  weeks  after  course  completion. 
The  level  of  initial  learning,  which  is  a  prime  variable  in  resisting  forgetting  (Hurlock  & 
Montague,  1982),  depends  on  the  amount  and  quality  of  instructional  practice.  Therefore, 
the  quality  of  the  methods  used  for  instructing  these  critical  and  complex  tasks  and  the 
methods  used  to  test  whether  students  understand  the  concepts  underlying  them  need  to 
be  evaluated. 


Objectives 


The  objectives  of  this  effort  were  to  evaluate  the  quality  of  instructional  methods 
used  to  teach  the  complex  tasks  of  acoustic  analysis  and  classification  at  the  Common 
Core  course  and  to  recommend  course  changes  that  would  improve  initial  acquisition  and 
long-term  retention  of  the  skills  and  knowledge  required  to  perform  those  tasks. 
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APPROACH 


Selection  of  Evaluation  Criteria 


Hurlock  and  Montague  (1982)  reviewed  literature  concerning  the  retention  of  skills 
and  knowledge  and  identified  three  variables  that  influence  both  the  initial  acquisition  and 
the  retention  of  complex  cognitive  skills.  These  variables  are  listed  and  elaborated  on 
below: 


1.  Structural  organization  of  course  content. 

a.  Students  should  be  provided  with  a  mental  model  of  the  task  that 
emphasizes  relationships  between  facts,  rules,  procedures,  principles,  and  concepts 
underlying  the  task. 

b.  Presentation  of  the  instructional  material  should  enforce  the  coherency  of 
the  mental  model  to  facilitate  the  recall  of  facts,  rules,  principles,  and  concepts. 

c.  Each  content  area  presented  in  the  course  should  be  taught,  practiced,  and 
tested  as  a  skill  that  supports  the  primary  task. 

d.  Course  learning  objectives  should  require  the  student  to  perform  the  task 
skills,  not  recall  or  recognize  verbal  descriptions  of  the  task. 

2.  Quality  of  feedback  and  remediation. 

a.  Students  should  be  provided  with  explicit  feedback  about  their  performance. 

b.  Feedback  should  be  provided  on  all  subparts  of  the  task,  as  well  as  at 
procedure  end-points  or  as  a  result  of  performance  on  tests. 

c.  Feedback  should  include  error  diagnosis  consistent  with  the  student's  mental 
model  of  the  task. 

d.  Remediation  of  conceptual  weaknesses  identified  by  error  diagnosis  should 
be  specific  to  the  needs  of  the  individual  student. 

3.  Testing  methods  used  to  evaluate  student  performance. 

a.  The  type  of  test  must  be  appropriate  to  the  type  of  performance  required; 
that  is,  courses  that  teach  cognitively  complex  tasks  should  measure  student  progress  in 
terms  of  the  quality  of  conceptual  understanding  of  the  relationships  between  facts,  rules, 
principles,  and  concepts. 

b.  Test  questions  that  measure  student’s  conceptual  understanding  should  be 
administered  orally  in  a  fill-in  or  short-answer  format. 

c.  In  cases  where  administrative  constraints  preclude  oral  testing,  questions 
should  be  formatted  as  "paper  and  pencil"  simulations  that  reveal  the  quality  of  the 
student's  conceptual  understanding  and  promote  posttest  diagnosis  of  conceptual  errors. 

d.  Multiple-choice  tests  that  require  the  student  to  recognize  discrete  facts 
should  be  avoided. 


These  three  variables  were  selected  as  the  criteria  for  evaluating  the  quality  of 
instruction  of  complex  tasks  at  the  Common  Core  course.  These  variables  and  related 
research  are  decribed  in  the  following  paragraphs. 

Structural  Organization  of  Course  Content 

Navy  course  design  has  been  heavily  influenced  by  rules  and  procedures  of  instruc¬ 
tional  system  design  (ISD).  Althugh  the  various  models  for  developing  instructional 
matrials  recognize  the  importance  of  good  instruction,  instructional  technology  has  yet  to 
develop  effective  guidance  for  organizing  complex  subject  matter.  The  instructional 
quality  inventory  (IQI)  (Ellis,  Wulfeck,  £c  Fredericks,  1979)  provides  clear  and  detailed 
guidance  for  the  writer  or  lecturer  to  follow  when  presenting  small  units  of  knowledge 
(e.g.,  facts,  principles)  but  was  not  intended  to  provide  guidance  in  developing  the 
organizational  framework  for  the  entire  course.  The  lack  of  systematic  guidance  has 
resulted  in  the  development  of  courses  that  are  inadequate  for  teaching  cognitively 
complex  tasks. 

Bunderson,  Gibbons,  Olsen,  and  Kearsley  (1981)  discussed  the  problems  that  com¬ 
monly  occur  in  developing  skill-oriented  courses.  Initially,  expert  performance  of  a  task 
is  analyzed  to  produce  a  list  of  verbal  descriptions  of  the  task.  These  descriptions  become 
the  basis  of  the  written  instructional  objectives  that  form  the  backbone  of  the  course,  as 
well  as  the  lesson  ware  that  is  developed  to  support  the  learning  objectives.  After 
students  have  completed  the  course,  they  are  tested  on  their  ability  to  recall  or  recognize 
the  verbal  descriptions  of  the  task.  It  is  expected  that,  at  some  point,  the  student  will  be 
able  to  "discover"  a  model  of  the  expert's  performance  of  the  task  from  the  verbal 
descriptions  that  will  enable  him  or  her  to  perform  the  task. 

Smith  and  Reigeluth  (1982)  stated  that  at  least  part  of  the  inadequacy  of  training 
matrials  developed  to  teach  complicated  skills  could  be  attributed  to  the  choice  of  subject 
matter  and  to  the  organization  of  materials.  They  also  stated  that  many  courses 
compound  this  inadequacy  by  following  an  exhaustive,  detailed,  linear  presentation, 
apparently  with  the  objective  of  conveying  stores  of  detailed  information  for  later  recall 
and  application.  This  approach  ignores  the  important  relationships  betwen  topics,  requires 
bri  force  memorization,  overloads  processing  ability,  provides  no  context  for  retrieval, 
and  demotivates  through  tedium.  Evans  and  Braby  (1983),  in  an  assessment  of  37  Navy 
technical  courses,  found  that  as  much  as  50  percent  of  the  content  of  lessons  is  isolated 
information  not  applied  in  follow-on  lessons.  They  suggest  that  many  courses  are 
structured  so  as  to  exclude  distributed  practice  in  application  of  newly  acquired  skill. 
This  could  account  for  low  retention  and  transfer  of  learned  skills  to  follow-on  courses 
and  to  the  job.  Because  of  the  course  organization,  students  are  required  to  memorize  a 
vast  number  of  discrete  facts  by  rote.  It  is  difficult  to  learn  and  retain  by  rote 
memorization.  Greeno  (1982)  stated  that,  before  learning  can  occur,  the  material  must  be 
related  to  some  general  structure  or  principle;  in  rote  learning,  the  new  procedure  or 
information  is  simply  associated  with  the  specific  problem  situation  in  which  it  is 
experienced.  Training  that  requires  students  to  memorize  large  amounts  of  facts  will 
result  in  poor  performance  because  their  conceptual  understanding  of  the  task  will  be 
limited  and  their  ability  to  generalize  rules  to  novel  situations  reduced. 

The  type  of  training  best  suited  to  teach  complex,  cognitive  knowledge  and  skills 
emphasizes  conceptual  understanding  of  the  task.  Smith  and  Reigeluth  (1982),  in  their 
development  of  a  structural  strategies  model,  emphasized  the  need  for  overall  course 
development  based  on  the  type  of  task  that  is  to  be  trained.  For  tasks  that  require 
students  to  learn  a  set  of  underlying  principles  needed  to  interpret  phenomena  or  solve 
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problems,  they  recommended  that  the  student  be  provided  with  a  framework  or  model 
that  emphasizes  the  causal  relationships  between  principles,  concepts,  and  procedures. 
For  instruction  to  be  effective,  the  conceptual  model  that  is  provided  must  promote  the 
student's  ability  to  explain,  predict,  or  diagnose  a  large  number  of  events,  problems,  or 
effects  (i.e.,  as  encountered  in  passive  acoustic  analysis,  electronic  countermeasures, 
manuevering).  Within  the  conceptual  framework  of  the  model,  the  student  can  be  taught 
the  principles  that  explain  "how"  or  "why"  an  event  occurs  and  then  be  given  practice  in 
applying  the  principles  in  a  number  of  situations.  Because  the  student  can  be  exposed  to 
only  a  limited  number  of  situations  in  a  training  environment,  the  training  must  be 
organized  so  that  the  student  thoroughly  understands  the  rules,  principles,  and  concepts 
underlying  the  task,  thus  ensuring  his  or  her  proper  performance  when  novel  events  or 
problems  are  encountered. 

Bunderson  et  al.  (1981)  promote  the  use  of  a  similar  strategy.  They  describe  a  "work 
model,"  where  the  student  can  practice  performing  the  task  under  conditions  that  are 
contextually  much  closer  to  "real  work"  conditions  than  are  the  verbal  descriptions 
provided  in  many  classrooms.  Stevens  and  Collins  (1978)  also  support  the  use  of 
conceptual  or  "mental"  models  in  instruction.  They  consider  mental  models  of  physical 
properties  to  be  important  in  understanding  complex  systems  and  emphasize  that  those 
models  must  represent  those  systems  accurately.  They  suggest  that  models  must  be 
powerful  enough  to  allow  the  student  to  consider  alternatives  and  derive  predictions  about 
novel  situations.  The  student  should  be  able  to  look  at  alternative  situations  by  running 
the  model  with  different  values  assigned  to  its  variables  to  help  him  or  her  understand  the 
concepts  underlying  the  device  or  process  the  model  is  representing. 

Conceptual  models  like  those  described  above  have  been  found  to  be  important  in  the 
learning  of  complex  tasks.  For  example,  Greeno  (1982)  showed  the  importance  of 
designing  instruction  to  teach  geometry  students  to  learn  the  underlying  concepts  needed 
to  represent  a  problem  graphically  before  applying  the  rule-based  procedures  to  solve  it. 
Substantial  variation  was  found  among  student  performance  in  solving  new  problems  that 
had  the  same  structure  as  those  they  had  previously  learned  to  solve.  Students  who  had 
been  taught  the  underlying  concepts  of  such  problems  were  better  able  to  solve  the  new 
ones  than  were  those  who  had  been  taught  the  procedures  without  explicitly  being  shown 
the  structure  of  relations  in  the  problems.  Rumelhart  and  Norman  (1980),  in  documenting 
the  progress  students  made  in  learning  a  computer  editing  system,  demonstrated  the  types 
of  errors  resulting  from  faulty  mental  models  of  computer  behavior.  Students  who  did  not 
have  a  good  mental  model  of  computer  functions  were  more  likely  to  make  errors  and  less 
likely  to  be  able  to  generate  a  reasonable  approach  to  correcting  those  errors  than  were 
others. 

Quality  of  Feedback/Remediation 

There  is  a  considerable  body  of  research  documenting  the  importance  of  receiving 
quality  performance  feedback  when  new  skills  and  knowledge  are  being  acquired  and  for 
maintaining  those  skills.  Feedback  can  range  from  providing  simple  corrections  on  tests 
to  providing  detailed  diagnostic  evaluation  of  the  student's  conceptual  errors.  Branson, 
Rayner,  Cox,  Furman,  King,  and  Hannum  (1975)  stated  that  providing  students  with 
feedback  about  their  performance  is  a  fundamental  concept  in  the  development  of 
instruction.  They  suggested  that  the  effect  of  feedback  on  learning  is  best  when  it  can  be 
provided  continuously  throughout  the  task  performance.  In  addition,  feedback  should  be 
provided  with  sufficient  diagnosis  of  errors  to  enforce  the  coherency  of  the  mental  model 
that  is  appropriate  to  the  task  the  student  is  trying  to  learn.  According  to  Branson  et  al., 
feedback  is  not  as  effective  when  it  is  provided  only  at  end-points  of  task  performance  or 
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as  results  of  tests,  because  it  does  not  provide  the  learner  with  the  explicit  information  as 
to  what  can  be  done  to  improve  performance. 

Briggs  (1977)  also  emphasizes  the  diagnostic  use  of  feedback  for  remediating  student 
errors  in  conceptually-based  tasks.  He  states  that,  when  a  student's  performance  is  only 
partly  adequate,  feedback  needs  to  be  given  to  define  the  kind  of  additional  study  needed 
to  improve  it.  This  type  of  feedback  is  important  because  it  explicitly  tells  the  student 
where  his  or  her  conceptual  understanding  is  weak  and  provides  appropriate  practice  to 
correct  that  weakness.  Cropper  (1980)  notes  that,  under  certain  conditions,  students  can 
generate  feedback  about  their  own  performance.  They  can  most  easily  generate  feedback 
when  performing  tasks  that  depend  on  the  correct  sequencing  of  steps  that  do  not  require 
the  application  of  learned  principles  for  correct  completion.  In  these  types  of  tasks,  the 
failure  to  complete  the  procedure  is  evid.at  to  the  student  (i.e.,  equipment  does  not 
function,  displays  are  not  available).  Self-generated  feedback  on  cognitively  complex 
tasks  would  require  that  the  student  thoroughly  understand  the  concepts  underlying  the 
task.  This  type  of  feedback  would  require  the  students  to  have  a  model  with  which  to 
compare  their  own  responses  to  determine  the  correctness  of  performance. 

Testing  Methods  Used  to  Evaluate  Student  Performance 

The  types  of  tests  selected  to  evaluate  student  performance  must  be  appropriate  to 
the  type  of  performance  being  tested.  Tests  of  student  performance  in  courses  that 
require  students  to  acquire  complex  knowledge  and  procedures  must  emphasize  the  level 
of  conceptual  knowledge  gained  by  the  student.  However,  such  tests  are  not  used  much  in 
Navy  technical  schools.  There  are  problems  associated  with  both  the  kinds  of  tests  used 
and  the  interpretation  of  student  performance  on  the  tests.  Weiss  (1979)  points  out  that 
the  testing  procedures  used  are  almost  invariably  those  based  on  a  testing  technology 
developed  in  the  1920s.  Almost  without  exception,  paper-and-pencil  tests  are  used  to 
measure  student  performance.  An  example  of  this  problem  in  the  Navy  is  the  emphasis  on 
the  use  of  the  multiple-choice  test  format,  which  evaluates  students'  performance  on  the 
basis  of  their  ability  to  recognize  discrete  facts  (NAVEDTRA  110;  NAVSEA  OD  45519). 
These  tests  continue  to  be  used,  no  matter  how  irrelevant  performance  on  multiple- 
choice,  fact-recognition  questions  is  to  the  kind  of  performance  required  on  the  job, 
primarily  because  of  ease  of  test  administration  and  familiarity  with  the  format. 

Much  of  the  performance  that  needs  to  be  assessed  on  complex,  cognitive  tasks  is  the 
level  of  conceptual  understanding  attained  by  the  student.  Cornbach  (1949)  suggests  that 
tests  that  assess  levels  of  understanding  often  are  not  used  because  facts  and  procedures 
loom  so  large  in  the  usual  classroom  that  teachers  and  test  designers  have  emphasized 
them  out  of  proportion  to  other  types  of  outcome.  He  points  out  that,  although  it  is 
important  to  measure  gains  in  knowledge  and  skill,  a  student  may  earn  a  high  score  on  a 
test  covering  only  memory-type  material  and  yet  have  made  little  progress  toward 
acquiring  a  good  cognitive  organization  of  that  knowledge.  The  problem  with  limiting  the 
assessment  to  tests  of  factual  knowledge  is  that  facts  that  are  little  understood  are 
quickly  forgotten.  Tests  of  ability  to  apply  principles  do  not  require  the  student  to  state 
or  recognize  a  fact  but,  rather,  to  solve  a  new  problem  in  which  that  principle  is  relevant. 
If  the  student  can  determine  the  correct  solution  to  a  new  problem  that  has  not  been 
studied  and  defend  that  solution  with  sound  principles,  it  is  more  likely  that  he 
"understands"  the  principle  underlying  that  problem. 

Recent  work  by  Ellis  and  Wulfeck  (1982)  provides  more  specific  guidance  in  selecting 
appropriate  test  item  format  for  the  type  of  performance  being  tested.  They  recommend 
that  fill-in  or  short-answer  items  be  used  for  testing  conceptual  understanding  of  complex 


tasks.  These  questions  should  be  administered  orally  rather  than  in  written  format  to 
avoid  problems  of  misinterpretation  as  well  as  to  allow  the  instructor  to  probe  for  greater 
detail  if  the  student's  answer  is  not  complete.  If  this  procedure  cannot  be  used  because  of 
administration  problems  or  time  constraints,  they  recommend  a  special  form  of  multiple- 
choice  questions  called  a  "paper-and-pencil"  simulation.  These  types  of  questions,  if  the 
incorrect  alternatives  are  constructed  carefuly,  can  effectively  diagnose  problems  in  the 
student's  logic  at  arriving  an  answer.  It  is  implicit  in  this  recommendation  that  the 
student's  incorrect  responses  be  used  to  diagnose  and  correct  the  conceptual  misunder¬ 
standing  that  caused  the  error. 

Description  of  Common  Core  Acoustic  Analysis  Course 

This  section  describes  the  Common  Core  Acoustic  Analysis  Course  in  terms  of  the 
three  criteria  described  above. 

Structural  Organization  of  Course  Content 

The  course  includes  instruction  on  five  separate  knowledge  topics  and  two  pro¬ 
cedures.  The  knowledge  topics  are  (1)  sonobuoys  (types,  uses,  and  characteristics),  (2) 
oceanography  (sound  channels,  the  velocity  of  sound,  and  the  physics  of  sound),  (3)  sources 
of  sound,  (4)  sound  source  signature  characteristics,  and  (5)  acoustic  intelligence  param¬ 
eters  for  specific  classes  and  types  of  ships.  The  two  procedures  are  (l)a  systematic 
acoustic  analysis  procedure  that  provides  the  student  with  guidelines  for  interpreting 
sound  source  information  on  the  lofargram,  and  (2)  a  computational  procedure  that  is 
comprised  of  a  number  of  mathematical  equations  and  that  provides  a  method  to  compute 
relationships  among  sound  sources  as  they  are  represented  on  the  lofargram. 

The  course  is  structured  such  that  the  first  four  knowledge  topics  and  both 
procedures  are  presented  to  the  student  during  the  first  week;  and  the  last  knowledge 
topic  (acoustic  intelligence  parameters  for  specific  classes  and  types  of  ships),  during  the 
last  3  weeks.  This  information  is  presented  in  a  linear  format,  with  acoustic  intelligence 
parameters  for  each  class  of  ships  being  detailed  on  separate  days.  Generally,  students 
listen  to  lectures  on  one  or  two  ship  classes  in  the  morning  and  attend  a  follow-up  gram 
analysis  practice  lab  specific  to  that  day's  lecture  topic  in  the  afternoon.  In  addition, 
they  spend  some  time  each  day  in  math  labs  that  allow  them  to  practice  using  the 
equations  from  the  computational  procedure  appropriate  to  the  specific  intelligence 
parameters  learned  that  day. 

A  review  of  the  course  objectives  revealed  that  each  segment  has  a  terminal 
objective  that  requires  the  student  to  "recall"  factual  information  (e.g.,  specific  intel¬ 
ligence  parameters,  operating  charateristics  of  equipment,  or  acoustic  signature  char¬ 
acteristics)  about  the  material  presented  in  the  lectures.  However,  in  actual  practice,  the 
student  only  has  to  recognize  the  correct  answer  on  a  multiple-choice  test.  Furthermore, 
a  partial  criterion  is  imposed  on  test  performance  accuracy:  The  student  is  required  to 
recognize  only  75  percent  of  the  factual  answers  correctly.  The  terminal  objectives  for 
the  procedural  skills  require  that  the  students  be  able  to  demonstrate  their  ability  to 
perform  the  procedures  as  tested  on  multiple-choice  knowledge  tests.  None  of  the  course 
objectives  require  the  student  to  be  able  to  perform  lofargram  analysis  in  an  integrated 
fashion  on  a  performance  test  or  to  classify  a  target  using  lofargram  analysis  skills. 
Therefore,  there  are  no  objectives  that  require  trainees  to  use  the  factual  or  task 
descriptive  information  they  have  learned  to  solve  typical  problems  likely  to  be 
encountered  on  the  job. 


Quality  of  Feedback /Remediation 

The  students  are  provided  feedback  in  a  number  of  ways: 

1.  They  are  encouraged  to  ask  questions  during  the  lectures.  The  instructors  make 
every  effort  to  answer  those  questions  during  the  class  period.  If  they  cannot  answer  the 
question,  they  refer  the  question  to  other  instructors. 

2.  Because  of  the  low  instructor/student  ratio,  opportunity  exists  for  one-to-one 
interaction  during  the  math  and  gram  labs. 

3.  The  students  receive  feedback  about  performance  on  weekly  tests  on  knowledge 
factors,  computational  skills,  and  gram  analysis  and  classification  skills. 

Remediation  is  made  available  to  students  under  two  conditions.  Students  who  are 
not  able  to  pass  the  tests  and  have  been  approved  by  the  school  review  board  are  usually 
"set  back"  and  allowed  to  take  the  course  over.  Students  who  meet  the  "pass"  criterion, 
but  whose  test  scores  are  low,  are  included  in  the  mandatory  study  group  and  required  to 
attend  2  hours  of  extra  class  per  day.  However,  during  those  2  hours,  they  do  not  review 
the  material  that  was  tested  by  the  exam  but,  rather,  the  material  presented  in  that  day's 
lecture.  No  specific  remediation  course  material  has  been  developed  to  train  students 
who  have  been  identified  as  low  performers. 

Testing  Methods  Used  to  Evaluate  Student  Performance 

The  student's  progress  on  course  objectives  is  measured  by  four  tests,  usually  given  at 
the  end  of  each  week  of  the  course.  The  first  three  tests  examine  student  proficiency  on 
information  and  skills  gained  in  the  preceding  week.  The  fourth  is  the  course  final  and 
covers  both  knowledge  factors  and  gram  analysis  skills  gained  over  the  entire  course. 

The  three  weekly  tests  are  divided  into  two  subparts.  The  first  part  tests  students' 
performance  on  acoustic  analysis  and  classification  on  10  lofargrams.  The  students  are 
required  to  annotate  the  grams,  demonstrate  the  use  of  the  systematic  analysis  procedure, 
and  integrate  the  acoustic  intelligence  appropriately  to  derive  the  correct  target 
classification.  The  second  part  is  a  40-question,  multiple-choice  test  that  evaluates 
students'  recognition  of  knowledge  factors.  This  test  also  includes  word  problems  that 
test  the  students'  ability  to  use  the  computational  formulas  and  input  appropriate  acoustic 
intelligence  information. 


RESULTS  AND  DISCUSSION 

Structural  Organization  of  Course  Content 

Four  major  probems  were  identified  concerning  the  course's  structural  organization: 

1.  There  is  no  clearly  defined  conceptual  model  that  promotes  the  students' 
understanding  of  the  important  relationships  between  topics.  This  lack  of  a  conceptual 
model  of  the  task  reduces  the  students'  ability  to  recall  or  infer  needed  information 
because  they  do  not  understand  the  relationships  among  the  facts,  rules,  principles,  and 
concepts  underlying  the  task. 


2.  The  information  is  presented  in  a  linear  fashion  with  the  emphasis  on  topics  or 
discrete  knowledge  factors,  forcing  the  students  to  spend  most  of  their  energy  on  rote 
memorization  of  a  large  number  of  facts. 

3.  There  is  no  practical  application  of  information  learned  in  the  oceanography  and 
sonobuoy  segments  of  the  course.  Although  these  topics  add  considerably  to  the  students' 
memory  load  requirements,  no  realistic  practice  occurs  that  would  result  in  useable  and 
measureable  skills.  It  would  be  more  appropriate  to  include  these  segments  at  a  point  in 
the  training  pipeline  where  the  information  could  be  applied  and  practiced. 

4.  The  course's  learning  objectives  do  not  require  students  to  perform  the  skill  of 
acoustic  analysis  and  classification  on  the  lofargram. 

Although  it  is  not  explicit,  the  real  objective  of  the  present  course  is  to  teach 
students  to  identify  targets  displayed  on  the  lofargram  using  acoustic  analysis  and 
classification  procedures.  Acoustic  analysis  skill  is  largely  based  upon  a  conceptual 
understanding  of  the  relationship  between  sound  generated  from  various  sources  (i.e., 
engines,  propellers,  and  auxiliary  sources)  and  the  representation  of  that  sound  on  the 
lofargram.  This  skill  is  not  presently  being  taught  so  that  the  student  can  accurately 
predict,  for  example,  how  changes  occurring  within  the  propulsion  system  will  affect 
representations  that  occur  on  the  lofargram.  Sound  source  characteristics  are  taught 
during  the  first  week  of  school,  and  students  are  shown  diagrams  of  the  various  propulsion 
systems.  However,  there  is  no  continual,  systematic  demonstration  of  the  intractive 
relationship  between  the  two  domains. 

To  promote  a  systematic  approach,  students  should  be  provided  with  a  mental  model 
of  the  task  that  emphasizes  the  relationships  between  sound  sources  and  their  representa¬ 
tions  on  the  lofargram.  This  model  should  be  based  on  the  mechanical  and  mathematical 
relationships  among  the  components  within  the  propulsion  systems  and  their  relationship 
to  other  target  sound  sources  that  will  allow  generation  of  the  appropriate  lofargram 
signature.  It  should  provide  a  conceptual  understanding  of  (1)  propulsions  systems  and 
other  sound  source  generators,  (2)  the  mathematical  relationships  among  components  of 
the  propulsion  system,  (3)  the  ways  in  which  the  propulsion  system  and  other  sound  source 
generators  are  represented  on  the  lofargram,  and  (4)  how  changes  in  the  propulsion  system 
will  effect  the  lofargram  signature. 

There  are  a  number  of  advantages  to  the  student  associated  with  using  a  model  that 
meets  these  criteria. 

1.  The  student  will  have  a  much  better  way  of  interpreting  the  acoustic  information 
on  the  lofargram  because  he  or  she  will  better  understand  the  sound  source  that  generated 
that  information. 

i  2.  Because  this  model  is  based  upon  the  mechanical  and  mathematical  relationships 

j  among  the  components  of  the  propulsion  system,  the  student  can  manipulate  the 

component  variables  (i.e.,  number  of  blades,  prime  mover  characteristics,  type  of  drive 
system)  to  generate  alternative  examples  of  the  class  of  ships  being  studied.  This  will  be 
particularly  valuable  in  understanding  the  lofargram  representation  of  ships  with  multiple 
engines  and  drive  systems,  as  well  as  in  showing  the  relationship  of  auxiliaries  to 
propulsion  system  changes. 

| 

f 

J 


8 


3.  The  model  should  provide  the  student  with  a  better  information  structure  for 
remembering  the  acoustic  parameters  associated  with  particular  types  of  ships  and  reduce 
the  need  for  rote  memorization  of  those  facts. 

This  model  should  also  be  useful  to  the  instructor  staff.  Presently,  demonstration  of 
lecture  points  about  representations  on  lofargrams  is  limited  by  a  small  training  gram 
library.  Often  the  grams  used  for  demonstration  do  not  clearly  and  uniquely  reinforce  the 
curriculum.  This  model  should  provide  the  instructor  with  a  way  to  demonstrate 
systematically  the  changes  on  the  lofargram  due  to  variables  such  as  propulsion  system 
component  differences,  speed,  and  depth,  as  well  as  provide  a  way  to  compare  aspects  of 
different  but  similar  classes  of  ships.  It  is  likely  that  one  of  the  most  useful  applications 
of  this  model  would  be  to  provide  the  instructors  with  a  method  for  diagnosing  conceptual 
errors  being  made  by  the  students.  These  errors  should  be  more  easily  corrected  by 
demonstrating  the  error  as  being  in  conflict  with  the  concepts  that  underlie  the  model. 
Tests  could  also  be  developed  that  would  be  aimed  toward  assessing  the  qualitative 
understanding  of  the  course  content  as  well  as  recall  of  acoustic  intelligence  and 
appropriate  use  of  analysis  procedures.  This  model  should  be  used  throughout  the  course 
to  gain  initial  conceptual  understanding  of  the  acoustic  analysis  procedures  and  to 
reinforce  those  concepts  while  teaching  parameters  of  specific  ship  types. 

The  learning  of  the  conceptual,  "mental"  model  could  be  facilitated  by  computer- 
based  instruction  (CB1).  A  mathematical  model  could  serve  as  the  basis  for  an  intractive, 
generative  CBI  system.  The  capability  to  use  the  model  to  generate  grams  would  provide 
a  more  systematic  demonstration  of  the  effects  of  individual  variables  that  affect  the 
iofargnm  signature.  The  ways  in  which  the  student  could  interact  with  the  materials 
could  also  be  more  varied.  In  addition,  a  significant  reduction  in  both  programming  time 
and  system  size  would  be  realized  because  of  the  systematic  approach  afforded  by  use  of 
this  model. 

Quality  of  Feedback/Remediation 

Although  the  instructor  staff  makes  a  concerted  effort  to  provide  performance 
feedback  to  the  students,  there  are  three  major  problems  in  the  way  feedback  and 
remediation  are  administered: 

1.  The  quality  of  diagnostic  feedback  for  repeated  errors  in  subprocedures  is 
inadequate.  Feedback  and  correction  of  conceptual  errors  is  not  performed  in  a 
systematic  way  that  ensures  the  student's  understanding  of  the  cause  of  error. 

2.  Students  have  no  way  of  evaluating  their  own  performance  largely  because  of  the 
lack  of  an  appropriate  mental  model  of  the  task. 

3.  There  is  no  systematic  approach  to  remediation  of  poorly  understood  knowledge 
or  procedural  skills. 

Feedback  about  the  correctness  of  performance  can  occur  in  a  number  of  ways. 
Feedback  given  to  students  for  performance  on  practice  exercises  and  tests  of  various 
types  is  often  limited  to  knowledge  of  the  number  of  correct  responses  to  items  included 
on  the  test,  with  varying  amounts  of  explanation  for  items  that  were  answered 
incorrectly.  In  tests  of  performance  on  complex  procedural  tasks,  errors  in  subprocedures 
may  receive  little  attention  in  the  test  review  process.  In  particular,  in  testing  situations 
observed,  little  or  no  diagnostic  feedback  was  given  to  students  concerning  repeated 
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errors  in  a  given  subprocedure  that  occurred  over  the  range  of  problems  presented  in  the 
test.  If  the  conceptual  error  that  caused  the  mistakes  in  the  subprocedure  is  not 
explicitly  identified  and  corrected,  the  student  is  likely  to  make  the  same  mistakes  in  the 
future. 

The  instructors  in  this  course  make  a  concerted  effort  to  provide  feedback  to 
students  while  in  class,  during  the  gram  labs,  and  following  the  tests.  It  would  be  difficult 
for  them  to  provide  high  quality,  individual  feedback  in  the  gram  labs  about  repeated 
conceptual  errors  due  to  the  number  of  students  and  the  fact  that  grams  are  evaluated  as 
they  are  completed  and  not  as  an  individual  package.  The  weekly  tests  provide  a  better 
framework  for  this  type  of  feedback.  Presently,  a  group  of  instructors  grades  the  grams 
as  they  are  finished  by  the  students,  primarily  to  save  time,  as  there  are  10  grams  per 
student  and  from  12  to  15  students.  To  provide  the  kind  of  quality  feedback  needed,  a 
single  instructor  should  review  the  packet  of  grams  for  individual  students  to  discover 
those  repeated  errors.  Although  students  review  the  grams  in  an  after-test  session,  they 
are  not  likely  to  discover  these  types  of  errors  by  themselves  and  be  able  to  correct  their 
mistakes. 

Students  scoring  a  low  passing  grade  on  one  of  the  weekly  tests  are  assigned  to  the 
mandatory  night  study  group  where  they  spend  an  additional  2  hours  per  day  in  studying 
models.  However,  they  do  not  review  the  material  on  which  they  scored  a  low  grade  but, 
rather,  the  material  presented  earlier  that  day.  The  rationale  is  based  on  the  idea  that 
students  with  low  passing  grades  simply  need  to  be  exposed  to  material  for  a  longer  time 
than  do  those  with  higher  grades.  The  real  problem  is  that  the  conceptual  errors  that 
caused  the  students'  poor  performance  are  not  identified  and  corrected. 

Some  way  is  needed  to  provide  students  remediation  when  they  do  not  completely 
understand  material,  particularly  when  repeated  errors  indicate  conceptual  misunder¬ 
standing.  The  students  could  be  provided  with  practice  time  on  specific  procedures.  One 
suggestion  would  be  to  develop  gram/math  lab  packages  for  use  by  students  who  need 
additional  practice  on  those  procedures.  If  the  students  make  a  large  number  of  errors 
involving  incorrectly  recalled  acoustic  intelligence  parameters,  the  information  structure 
used  by  the  student  could  be  reviewed  to  provide  some  insight  to  the  problem.  Those 
students  could  be  helped  in  developing  and  understanding  that  information  structure,  as 
well  as  allowed  time  to  practice  the  recall  of  those  items.  It  would  probably  be  helpful  to 
all  students  if  time  were  set  aside  in  class  for  practice  of  recall  of  those  parameters. 
Although  the  classification  procedure  requires  the  accurate  recall  of  acoustic  intelligence 
parameters,  the  classified  status  of  those  parameters  makes  it  impossible  for  students  to 
study  them  at  night  on  their  own;  further,  each  day's  lecture  adds  considerably  to  the 
material  that  needs  to  be  remembered.  Unless  a  specific  time  is  set  aside  to  review  the 
material,  students  will  find  it  difficult  if  not  impossible  to  recall  specific  parameters 
accurately. 

Testing  Methods  Used  to  Evaluate  Student  Performance 

The  primary  problem  in  the  area  of  knowledge  testing  is  that  the  students'  progress  in 
meeting  course  objectives  is  measured  by  their  ability  to  recognize  discrete  facts  in  a 
multiple-choice  format,  rather  than  demonstrate  whether  they  understand  fundamental 
relationships  by  solving  typical  problems.  The  emphasis  of  the  testing  does  not  allow  for 
diagnostic  analysis  of  conceptual  misunderstanding. 


Most  of  the  questions  on  the  present  knowledge  test  are  aimed  at  testing  the 
student's  ability  to  recognize  discrete  facts  (i.e.,  number  of  blades,  type  of  prime  mover, 
type  of  drive  system).  Because  of  the  conceptual  nature  of  this  task,  questions  that  probe 
the  student's  conceptual  understanding  should  be  included.  Examples  of  those  types  of 
questions  might  be: 

1.  How  will  a  change  in  a  particular  drive  system  affect  the  lofargram  signature  on 
a  specified  contact? 

2.  How  will  an  increase  in  speed  affect  the  various  components  of  the  propulsion 
system  as  represented  on  the  lofargram? 

3.  How  will  a  change  in  the  gear  reduction  ratio  affect  the  relationship  between  the 
prime  mover  and  the  propellers? 

The  student's  answers  to  these  types  of  questions  will  not  only  demonstrate  the  level  of 
the  student's  understanding  but  also  provide  the  instructors  with  a  good  diagnostic  tool  to 
better  understand  the  problems  that  the  student  is  having  with  the  material. 

The  types  of  tests  used  to  evaluate  the  level  of  student  knowledge  of  discrete  facts 
(e.g.,  specific  items  of  acoustic  intelligence)  are  limited  to  the  multiple-choice  test 
format.  Students  are  tested  on  their  ability  to  recognize  correct  answers  contained  in  a 
group  of  possible  answers  rather  than  recall  the  appropriate  information  from  memory. 
The  testing  is,  therefore,  inconsistent  with  the  objectives.  When  these  students  are 
performing  tasks  on  the  job,  they  will  be  required  to  recall  this  information,  not  r  cognize 
it  from  a  group  of  possible  answers.  Therefore,  it  would  be  better  to  evaluate  the 
student's  performance  by  using  the  job  standard  of  recall  rather  than  the  multiple-choice 
standard  of  item  recognition.  This  could  be  accomplished  by  using  questions  that  require 
fill-in  answers,  short  answers,  or  listing  of  facts.  Multiple-choice  questions  could  be  used 
to  assess  student  performance  on  computational  questions  if  the  alternative  answers  are 
designed  to  reveal  weaknesses  in  the  procedures  or  acoustic  intelligence  recall  required  to 
solve  the  problem.  However,  these  types  of  questions  should  be  used  only  if  the  intent  is 
to  diagnose  student  errors  on  these  problems  and  provide  the  student  with  corrective 
feedback. 


CONCLUSIONS 

1.  The  instructors  and  the  staff  at  the  Common  Core  course  are  both  hard-working 
and  consciencious:  They  made  every  effort  to  teach  students  to  become  competent 
acoustic  analysts.  Although  this  evaluation  revealed  a  number  of  deficiencies  in  course 
organization,  diagnostic  feedback,  and  testing  methods,  instructor  attitude  is  not  cited  as 
a  problem.  It  is  very  likely  that  these  deficiencies  have  been,  in  part,  overcome  by 
individual  instructor  efforts.  However,  relying  on  this  type  of  unsystematic  approach 
cannot  ensure  high  quality  graduates  of  this  school. 

2.  The  Common  Core  course,  as  it  is  presently  being  taught,  does  not  explicitly 
provide  the  student  with  a  qualitative  understanding  of  the  relationship  between  the  sound 
source  generators  in  the  propulsion  system  and  their  representations  on  the  lofargram. 
The  development  and  use  of  a  model  should  (a)  provide  a  better  conceptual  understanding 
of  this  relationship  that  will  aid  the  student  in  the  initial  acquisition  of  the  skills  and 
knowledge  underlying  acoustic  analysis  and  classifiation  techniques,  (b)  provide  a  better 


I' 


Y 


information  structure  for  the  retention  of  the  procedural  skills  and  the  memorized 
acoustic  intelligence  data  base,  and  (c)  aid  the  instructors  in  curriculum  development, 
classroom  demonstrations,  and  test  development. 

3.  Although  the  student  is  provided  with  feedback  about  performance  on  knowledge 
and  procedural  skills,  little  individual  feedback  is  directed  toward  diagnosing  and 
correcting  repeated  errors  caused  by  conceptual  misunderstandings.  This  type  of 
feedback  could  be  provided  by  an  individual  instructor  reviewing  test  results,  rather  than 
by  a  group  of  instructors  grading  the  lofargrams  from  many  different  students.  Feedback 
of  this  quality  should  aid  the  student  in  understanding  the  underlying  concepts  of  the  task. 

4.  Tests  of  knowledge  factors  do  not  presently  require  the  students  to  demonstrate 
conceptual  understanding  of  the  material.  Without  these  types  of  questions,  students  may 
be  encouraged  to  employ  rote  memorization  learning  techniques.  Although  such  tech¬ 
niques  enable  students  to  perform  adequately  on  a  test,  they  may  limit  their  ability  to  use 
the  information  to  perform  well  at  a  later  time.  In  addition,  tests  are  presently 
constructed  to  test  the  students'  recognition  rather  than  recall  of  material  presented 
during  the  course.  The  students  would  be  better  prepared  for  the  job  if  they  were  tested 
on  their  ability  to  recall  specific  information. 


RECOMMENDATIONS 

It  is  recommended  that  FASOTRAGRUPAC  support  the  following  actions: 

1.  Improve  the  Common  Core  course  by  the  development,  evaluation,  and  use  of  a 
model  designed  to  increase  the  students'  conceptual  understanding  of  the  relationship 
between  sound  sources  and  the  representations  of  those  sounds  on  the  lofargram.  The 
course  curriculum  would  be  developed  around  those  principles  and  incorporate  the  model 
of  the  task. 

2.  Ensure  that  students  use  the  model  to  structure  acoustic  intelligence  parameters 
and  be  allowed  class  time  to  practice  the  recall  of  those  items. 

3.  Ensure  that  information  presented  in  the  oceanography  and  sonobouy  segments  of 
the  course  is  taught,  practiced,  and  tested  as  skills  needed  to  perform  acoustic  analysis 
and  classification.  If  practical  application  of  that  information  cannot  be  accomplished  in 
this  course,  those  segments  may  be  moved  to  a  more  appropriate  point  in  the  training 
pipeline. 


4.  Ensure  that  course  objectives  require  students  to  perform  the  skill  of  acoustic 
analysis  and  classification  of  the  lofargram. 

5.  Ensure  that  instructors  systematically  attend  to  problems  of  student  misunder¬ 
standing  and  provide  diagnostic  feedback  to  individual  students  about  the  conceptual 
errors  in  their  understanding  of  the  task  as  evidenced  by  repeated  errors  in  subprocedures. 

6.  Ensure  that  tests  assess  the  student's  level  of  conceptual  understanding  of  the 
materials  presented  in  the  course,  as  well  as  the  student's  ability  to  recall  rather  than 
recognize  factual  information. 

7.  Ensure  that  remediation  of  students  who  perform  poorly  on  these  tests  is 
directed  toward  correcting  the  poor  performance  rather  than  requiring  students  to  study 
new  material  for  a  longer  period  of  time. 
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